Upgrade Spanner to the latest version by sakthivelmanii · Pull Request #37433 · apache/beam

sakthivelmanii · 2026-01-27T16:40:53Z

Please add a meaningful description for your change here

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
Update CHANGES.md with noteworthy changes.
If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

gemini-code-assist · 2026-01-27T16:41:15Z

Summary of Changes

Hello @sakthivelmanii, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request refactors the dependency management for Google Cloud Spanner within the project's Gradle build system. Instead of explicitly declaring and pinning the Spanner version, the changes delegate this responsibility to the google_cloud_platform_libraries_bom. This approach streamlines versioning, promotes consistency across Google Cloud dependencies, and simplifies future updates by centralizing version control.

Highlights

Dependency Version Management: The explicit version definition for google_cloud_spanner_version (previously 6.104.0) has been removed from the build configuration.
Spanner BOM Integration: The google_cloud_spanner_bom and google_cloud_spanner_test dependencies are now implicitly managed by the google_cloud_platform_libraries_bom, ensuring version consistency with other Google Cloud Platform libraries.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

bharadwaj-aditya

Some minor changes

...io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerConfig.java

.../google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerAccessor.java

...gle-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/spanner/SpannerAccessorTest.java

damccorm · 2026-01-29T21:35:31Z

...ava/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerIO.java

-    }
+    SpannerAccessor sa = SpannerAccessor.getOrCreate(spannerConfigWithCredential);
+    DatabaseClient databaseClient = sa.getDatabaseClient();
+    return databaseClient.getDialect();


Is there a reason to undo this change? It seems like it is still probably better to force the SpannerAccessor class to close

damccorm · 2026-01-29T21:36:10Z

buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy

    def google_cloud_bigdataoss_version = "2.2.26"
    // [bomupgrader] determined by: com.google.cloud:google-cloud-spanner, consistent with: google_cloud_platform_libraries_bom
-    def google_cloud_spanner_version = "6.104.0"
+    def google_cloud_spanner_version = "6.107.0"


Do we need this version to be tracked at all now or can we remove this line?

.../google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerAccessor.java

damccorm · 2026-01-29T21:38:10Z

.../google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerAccessor.java

+        .map(ValueProvider::get)
+        .orElse(DEFAULT_SESSION_WAIT_DURATION);
+    builder.setSessionPoolOption(
+        SessionPoolOptions.newBuilder().setWaitForMinSessionsDuration(waitDuration).build());


Is this change related to the hangs?

No. This is related to DEADLINE EXCEEDED in dataflow workers. b/462499883

github-actions · 2026-02-03T10:48:13Z

Assigning reviewers:

R: @ahmedabu98 for label java.
R: @damccorm for label build.
R: @nielm for label spanner.

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

stop reviewer notifications - opt out of the automated review tooling
remind me after tests pass - tag the comment author after tests pass
waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

bharadwaj-aditya · 2026-02-03T13:21:15Z

...io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerConfig.java

+   * @return {@link SpannerConfig}
+   *     <p>Sets the wait time for multiplexed session to be available while creating a database
+   *     client. Setting this will block the {@link com.google.cloud.spanner.DatabaseClient}
+   *     creation. By default, We will be setting 5 mins as minimum wait time.


Link to the variable in these docs, just in case this changes at a later date, the docs need to be in Sync

bharadwaj-aditya

LGTM

damccorm

Thanks, LGTM. Just waiting on checks to succeed before merging.

damccorm · 2026-02-04T19:52:28Z

Please don't force push since that:

Resets all checks
Makes it difficult to tell what has been reviewed (GitHub no longer has an easy way to see the diff from my last review)

I'll retry tests if there are flakes

damccorm · 2026-02-04T20:26:04Z

Spotless and precommit java failures are unrelated, fixed by #37505

damccorm · 2026-02-04T21:33:26Z

The GCP IO one seems maybe related though:

SpannerChangeStreamOrderedWithinKeyGloballyIT > testOrderedWithinKey FAILED
    java.lang.AssertionError at SpannerChangeStreamOrderedWithinKeyGloballyIT.java:159
        Caused by: java.lang.AssertionError at MatcherAssert.java:18

@sakthivelmanii could you please take a look? This looks mostly green at head - https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_GCP_IO_Direct.yml?query=branch%3Amaster+event%3Aschedule

You can download the test results from https://github.com/apache/beam/actions/runs/21685692071?pr=37433

damccorm · 2026-02-05T14:09:09Z

I'm still seeing Spanner ChangeStreams test failures

SpannerChangeStreamOrderedWithinKeyGloballyIT > testOrderedWithinKey FAILED
    java.lang.AssertionError at SpannerChangeStreamOrderedWithinKeyGloballyIT.java:159
        Caused by: java.lang.AssertionError at MatcherAssert.java:18

SpannerChangeStreamOrderedByTimestampAndTransactionIdIT > testTransactionBoundaries FAILED
    java.lang.AssertionError at SpannerChangeStreamOrderedByTimestampAndTransactionIdIT.java:159
        Caused by: java.lang.AssertionError at MatcherAssert.java:18

Also, as mentioned above - please avoid force pushing, it makes reviewing the PR harder

sakthivelmanii · 2026-02-05T14:19:01Z

The same tests are failing in master branch as well.

sakthivelmanii · 2026-02-05T14:19:28Z

I had to rebase my branch with the latest master to make spotless tests to pass

damccorm · 2026-02-05T14:29:30Z

The same tests are failing in master branch as well.

https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_GCP_IO_Direct.yml?query=event%3Aschedule - it looks like testTransactionBoundaries is flaky sometimes on master, I don't see any failures involving testOrderedWithinKey on master though. In this PR it is red every time, so it seems like something is regressing.

I had to rebase my branch with the latest master to make spotless tests to pass

Note that you can do this by merging in the master branch without losing commit history (which is the thing that GitHub's review feature relies on). (git pull origin master)

sakthivelmanii · 2026-02-09T14:36:14Z

@damccorm I am seeing that setting SessionPoolOptions is causing the issue. I am trying to debug the issue. I will move this into two different PRs.

Spanner upgrade.
introduce an additional to configure session creation wait time.

damccorm · 2026-02-09T14:50:28Z

Sounds good, thanks - please comment once they are ready for review

github-actions · 2026-02-17T12:25:53Z

Reminder, please take a look at this pr: @ahmedabu98 @damccorm @nielm

damccorm · 2026-02-17T12:56:17Z

Closing this PR for now since it was split up

sakthivelmanii marked this pull request as draft January 27, 2026 16:40

github-actions bot added the build label Jan 27, 2026

github-actions bot added java io gcp labels Jan 27, 2026

sakthivelmanii force-pushed the master branch 2 times, most recently from b36d8fc to 636fccc Compare January 27, 2026 18:19

github-actions bot added the spanner label Jan 27, 2026

sakthivelmanii force-pushed the master branch from 636fccc to 789e4f4 Compare January 27, 2026 18:31

bharadwaj-aditya suggested changes Jan 28, 2026

View reviewed changes

sakthivelmanii force-pushed the master branch 3 times, most recently from dc8a042 to b930c89 Compare January 29, 2026 20:30

damccorm reviewed Jan 29, 2026

View reviewed changes

sakthivelmanii force-pushed the master branch 4 times, most recently from c938e2b to a61409b Compare January 30, 2026 06:42

sakthivelmanii marked this pull request as ready for review February 3, 2026 10:21

github-actions bot added the Next Action: Reviewers label Feb 3, 2026

bharadwaj-aditya reviewed Feb 3, 2026

View reviewed changes

sakthivelmanii force-pushed the master branch 3 times, most recently from cbdcee0 to 1bc651a Compare February 4, 2026 09:12

bharadwaj-aditya approved these changes Feb 4, 2026

View reviewed changes

damccorm approved these changes Feb 4, 2026

View reviewed changes

sakthivelmanii force-pushed the master branch from 1bc651a to 736e6d0 Compare February 4, 2026 19:39

sakthivelmanii added 2 commits February 5, 2026 13:35

Update spanner version and fix hanging issue

2325d11

add info logs

45e1d86

sakthivelmanii force-pushed the master branch from dc69dc1 to 45e1d86 Compare February 5, 2026 08:06

sakthivelmanii added 6 commits February 6, 2026 17:40

Merge remote-tracking branch 'upstream/master'

d3f3e7e

Merge remote-tracking branch 'upstream/master'

39511af

disable retry

c4c9f69

Set default session wait time to 0

7e13513

Remove info in the workflow

d116466

comment out setting session pool option

d4759e1

github-actions bot added the slow-review label Feb 17, 2026

damccorm closed this Feb 17, 2026

github-actions bot removed the slow-review label Feb 17, 2026

Conversation

sakthivelmanii commented Jan 27, 2026

GitHub Actions Tests Status (on master branch)

Uh oh!

gemini-code-assist bot commented Jan 27, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

bharadwaj-aditya left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

damccorm Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

damccorm Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

damccorm Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

sakthivelmanii Jan 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Feb 3, 2026

Uh oh!

bharadwaj-aditya Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

bharadwaj-aditya left a comment

Choose a reason for hiding this comment

Uh oh!

damccorm left a comment

Choose a reason for hiding this comment

Uh oh!

damccorm commented Feb 4, 2026

Uh oh!

damccorm commented Feb 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

damccorm commented Feb 4, 2026

Uh oh!

damccorm commented Feb 5, 2026

Uh oh!

sakthivelmanii commented Feb 5, 2026

Uh oh!

sakthivelmanii commented Feb 5, 2026

Uh oh!

damccorm commented Feb 5, 2026

Uh oh!

sakthivelmanii commented Feb 9, 2026

Uh oh!

damccorm commented Feb 9, 2026

Uh oh!

github-actions bot commented Feb 17, 2026

Uh oh!

damccorm commented Feb 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

sakthivelmanii Jan 30, 2026 •

edited

Loading

damccorm commented Feb 4, 2026 •

edited

Loading